Personal authentication apparatus and personal authentication method

ABSTRACT

A personal authentication apparatus comprises an input unit configured to input image data; a face detection unit configured to detect a face region of a person included in the image data input by the input unit, and to detect feature data from the detected face region; a facial expression determination unit configured to determine a facial expression from the face region detected by the face detection unit; a storage unit configured to store feature data used to authenticate a person in correspondence with respective facial expressions of a plurality of faces; a selection unit configured to select feature data corresponding to the facial expression determined by the facial expression determination unit from the storage unit; and an authentication unit configured to authenticate a person by comparing the feature data of the face region detected by the face detection unit, and the feature data selected by the selection unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a technique for detecting a face regionof a person from image data, and executing personal authentication.

2. Description of the Related Art

Conventionally, a technique for executing personal authentication usinga face detection function is known. This technique extracts feature dataof a face region different for each person from detected face imagedata, and compares the feature data with those which are registered inadvance, thereby identifying whether or not the detected face is that ofa registered person.

However, since the feature data are influenced by the facial expressionof the face region, the difference between the facial expression of theface region and that upon registration deteriorates the precision ofpersonal authentication.

In order to avoid this, a method of detecting the facial expression of aface region, determining whether or not the detected facial expressionis valid to execute personal authentication, and executing the personalauthentication using new image data when it is determined that thefacial expression is invalid is known (for example, see Japanese PatentLaid-Open No. 06-119433).

However, with the method described in Japanese Patent Laid-Open No.06-119433, since the personal authentication cannot be started until avalid facial expression is obtained, it takes much time untilauthentication. Also, when a valid facial expression does not appear,the authentication is disabled.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of theaforementioned problems, and implements a personal authenticationtechnique which can quickly execute personal authentication with higherprecision even when a facial expression changes.

In order to solve the aforementioned problems, the present inventionprovides a personal authentication apparatus comprising: an input unitconfigured to input image data; a face detection unit configured todetect a face region of a person included in the image data input by theinput unit, and to detect feature data from the detected face region; afacial expression determination unit configured to determine a facialexpression from the face region detected by the face detection unit; astorage unit configured to store feature data used to authenticate aperson in correspondence with respective facial expressions of aplurality of faces; a selection unit configured to select feature datacorresponding to the facial expression determined by the facialexpression determination unit from the storage unit; and anauthentication unit configured to authenticate a person by comparing thefeature data of the face region detected by the face detection unit, andthe feature data selected by the selection unit.

The present invention also provides a personal authentication method tobe executed by a personal authentication apparatus which includes: aninput unit configured to input image data; a face detection unitconfigured to detect a face region of a person included in the imagedata input by the input unit, and to detect feature data from thedetected face region; a facial expression determination unit configuredto determine a facial expression from the face region detected by theface detection unit; and a storage unit configured to store feature dataused to authenticate a person in correspondence with respective facialexpressions of a plurality of faces, the method comprises: a selectionstep of selecting feature data corresponding to the facial expressiondetermined by the facial expression determination unit from the storageunit; and an authentication step of authenticating a person by comparingthe feature data of the face region detected by the face detection unit,and the feature data selected in the selection step.

According to the present invention, even when a facial expressionchanges, since personal authentication is executed by selectingappropriate feature data according to the facial expression, theauthentication precision can be improved.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the arrangement of a personalauthentication apparatus according to the first embodiment of thepresent invention;

FIG. 2 is a view showing an example of a personal database according tothe first embodiment;

FIG. 3 is a flowchart showing personal authentication processingaccording to the first embodiment;

FIG. 4 is a block diagram showing the arrangement of a personalauthentication apparatus according to the second embodiment;

FIGS. 5A and 5B are a flowchart showing personal authenticationprocessing according to the second embodiment;

FIG. 6 is a view showing an example of features; and

FIG. 7 is a view showing an example of features in case of a “smile”face.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described in detailhereinafter with reference to the accompanying drawings.

Note that the embodiments to be described hereinafter are merelyexamples upon implementation of the present invention and should beappropriately modified or changed depending on the arrangements ofapparatuses and various conditions to which the present invention isapplied, and the present invention is not limited to the followingembodiments.

First Embodiment

FIG. 1 is a schematic block diagram showing the arrangement of apersonal authentication apparatus according to an embodiment of thepresent invention.

A personal authentication apparatus of this embodiment inputs image datafrom an image input unit 1, and a face detection unit 2 detects theposition and size of a face region of a person from this image data. Asfor face detection, a given method can be used. For example, the facedetection unit 2 extracts shapes corresponding to components of a faceregion such as a nose, mouth, and eyes from the input image data, anddetects a region where the nose and mouth exist on an extended line thatpasses through the center between the two eyes. The face detection unit2 estimates the size of the face based on the sizes of the two eyes andtheir distance, and specifies, as a face region, a region bounded by aregion having the estimated size with reference to a positioncorresponding to the center of the nose.

The face detection unit 2 extracts feature data from the detected faceregion. As disclosed in, for example, Japanese Patent Laid-Open No.2005-266981, the feature data include information associated withpractical shapes of components of a face such as a mouth, eyes,eyebrows, and nose, and the positions of these components. Note that thefeature data can be extracted by calculations from image data of theinput face region using, e.g., a neural network or an edge detectionmethod that uses a spatial filter. Of course, not only informationassociated with the shapes and positions but also information associatedwith saturations and hues may be included in the feature data. Theappearance of the face can be analyzed in more detail with increasingnumber of feature data per face, and the precision of the facialexpression determination and personal authentication using the featuredata can be improved.

In image data including a plurality of frames like a movie, faces aredetected for respective frames at given periods. An identical persondetermination unit 3 determines which faces that appear in these framescorrespond to an identical person. For example, when a plurality offaces are detected from image data of a certain frame, and one or aplurality of faces are detected from image data of another frame, theidentical person determination unit 3 determines that faces havingsimilar sizes and positions correspond to an identical person.

A facial expression determination unit 4 determines a facial expressionof a face based on the feature data extracted by the face detection unit2. The types of facial expressions to be determined include “blankness”,“smile”, and “eye closure”. A database selection unit 5 selects andreads out, based on the determination result of the facial expressiondetermination unit 4, feature data corresponding to that facialexpression from feature data for personal authentication, which areregistered in advance in a personal database unit 8. An authenticationunit 6 executes personal authentication by comparing and collating thefeature data selected by the database selection unit 5 with those of theface region extracted by the face detection unit 2, and a result outputunit 7 outputs the authentication result.

This personal authentication apparatus may be configured as a standaloneapparatus or a system including a plurality of apparatuses. For example,a standalone digital camera or digital video camera may include allcomponents from the image input unit 1 to the personal database unit 8.Alternatively, a digital camera or digital video camera may include onlythe image input unit 1, and an external computer that can communicatewith the digital camera or digital video camera may include othercomponents. Furthermore, a plurality of computers on a network may shareand include all components from the image input unit 1 to the personaldatabase unit 8, and the computer including the image input unit 1 mayreceive image data from another external apparatus or recording media.

In this embodiment, as feature data for personal authentication, thecoordinates of 23 features are used, as shown in FIG. 6. In order toexecute personal authentication in practice, more features are required.However, the following description will be made under the assumptionthat 23 features are used, for the sake of simplicity.

When a face in image data has a tilt, these 23 features are calculatedafter the image data is rotated to remove this tilt. The coordinates ofthese features are calculated based on the positions of the eyes, nose,mouth, eyebrows, and the like extracted from the image data by the facedetection unit 2 by normalizing the size of the face by, e.g., thedistance between the two eyes. This coordinate position is determinedwith reference to the position of the end point of the nose. Theauthentication unit 6 defines the coordinates of the features calculatedfrom the input image data by Pi (i=1, 2, . . . , 23), and calculates anabsolute value sum S=Σ|Pi−P′i| of differences from coordinates P′i offeatures of a person registered in advance in the personal database unit8. As this absolute value sum S is smaller, a person to be detected anda person registered in advance have a high possibility of an identicalperson. When the absolute value sum S of a person determined to have thehighest possibility is equal to or smaller than a pre-set threshold, theauthentication unit 6 determines that the person to be detected is aregistered person; otherwise, the authentication unit 6 determines thatthere is no corresponding person.

Note that this method of calculating the absolute value sum S is oneexample of personal authentication methods, and personal authenticationmay be executed using other methods. For example, a person may beidentified from change patterns of the positions and shapes of the eyesand mouth upon changing from a “blankness” face to a “smile” face, or alarge number of image data having different resolutions may be generatedfrom image data of a face, resolution-dependent personal authenticationprocesses may be executed, and a final personal authentication resultmay be obtained from the integrated result of these processes. That is,even when other methods are used, the same effects as in this embodimentcan be obtained as long as an arrangement in which a person with ahighest possibility is determined by collating with data registered inadvance in the personal database unit 8 is adopted.

Facial expression determination by the facial expression determinationunit 4 can be implemented by detecting temporal changes of relativepositions of the features on an identical face. For example, in case ofa “smile” face, the positions of the features are changed like that theeyes are narrowed, and the corners of the mouth are lifted up, as shownin FIG. 7, compared to a “blankness” face shown in FIG. 6. In thisembodiment, when the distance between features 9 and 11 of upper andlower central points of the observers' left eye, and the distancebetween features 13 and 15 of upper and lower central points of theobservers' right eye are decreased at a predetermined ratio or more withrespect to those on the “blankness” face, and the positions of features6 and 7 of the left and right ends of the mouse with respect to features4 and 5 of upper and lower central ends of the mouse are lifted up at apredetermined ratio or more compared to the “blankness” face, “smile” isdetermined. Furthermore, when the positions of the features 4 to 7 ofthe mouth do not satisfy the “smile” condition, and the distance betweenthe features 9 and 11 and that between the features 13 and 15 of theeyes are decreased compared to those on the “blankness” face, “eyeclosure” is determined.

As for determination of a face region of a “blankness” face, forexample, when the change amount of the shape of the mouse does notexceed a predetermined threshold for a predetermined period of time, aface region obtained at that time can be determined as a “blankness”state. Alternatively, values obtained by averaging the features ofseveral successive frames for an identical face region may be used todefine a “blankness” state. Alternatively, by comparing the featuresobtained over a plurality of successive frames for an identical faceregion, a frame including a face corresponding to “blankness” and thatincluding a face corresponding to “smile” may be automaticallydetermined based on their relative values. Furthermore, conditionsassociated with the degree of opening of the eyes and mouth shape may beset in correspondence with facial expressions, it may be determinedwhich of conditions a face of each input frame image satisfies, and afacial expression may be independently determined for each frame. Withthe arrangement that independently determines a facial expression foreach frame, a facial expression can be determined even from a stillimage as a single frame. In this manner, a facial expression isdetermined by determining whether or not the shapes of parts such as theeyes and mouse which specify a face and are extracted from image datasatisfy specific conditions, unlike in personal authentication whichmakes comparison with features registered in advance for each person.

As described above, in the personal authentication, the featuresdetected from image data are compared with those registered in advancein the personal database unit 8 for each person to calculate theirabsolute value sum S. However, when the facial expression of a personchanges, the coordinates of the features detected from the image dataalso change, and the value of the absolute value sum S calculated uponpersonal authentication largely varies depending on facial expressions,thus lowering the precision of the personal authentication. By contrast,the personal authentication apparatus of this embodiment can improve theprecision since it executes personal authentication as follows.

Feature data for personal authentication are registered in advance inthe personal database unit 8 in correspondence with persons whosepersonal authentication is to be executed, and their facial expressions,as shown in FIG. 2. At the time of registration, an object maysequentially create designated facial expressions, and feature dataextracted from respective captured facial expressions may be registered.Alternatively, facial expressions may be automatically determined by theaforementioned method from image data including facial expressionsarbitrarily changed by an object, and their features may be registered.Alternatively, the user may select an arbitrary face from alreadycaptured image data, and may register the features of that face asfeature data of a person and facial expression designated by the user.

The operation at the time of personal authentication will be describedbelow with reference to the flowchart of FIG. 3. This flowchart isstarted when image data is input to the image input unit 1. If the imageinput unit 1 includes a camera, this image data corresponds to thatwhich is captured by that camera or is read out from a recording medium.If the image input unit 1 includes a personal computer, the image datacorresponds to an image read out from a recording medium or image datareceived via a network. This image data may be either a still image ormovie. If the image data is a movie, personal authentication issuccessively executed at frame intervals according to a time periodrequired for the personal authentication. The following description ofthe embodiment will be given under the assumption that image data of amovie is input to the image input unit 1.

In step S101, the face detection unit 2 receives image data for oneframe of the movie from the image input unit 1, and detects faces ofpersons.

If it is determined in step S102 that the face detection unit 2 candetect one or more faces, the process advances to step S103; otherwise,the process jumps to step S111. It is determined in step S111 if theimage data input to the image input unit 1 includes image data ofanother frame. If the image data includes another frame, the image datais updated in step S112. Then, the process returns to step S101, and theface detection unit 2 executes face detection from the updated imagedata.

In step S103, the identical person determination unit 3 receives thedetection results of faces detected by the face detection unit 2, anddetermines which of faces detected from different frames are consideredas a face of an identical person. If image data includes a plurality ofpersons, their faces have to be distinguished from each other, andfeature data used as the aforementioned “blankness” criteria have to becalculated for each face. For this purpose, the identical persondetermination unit 3 compares the central positions and sizes ofrespective faces detected from respective frames, and estimates a facehaving the smallest sum total of the distances of the central positionof faces and size change amounts among successive frames as a face of anidentical person. However, even when this sum total is smallest, if itfalls outside a pre-set threshold range, the identical persondetermination unit 3 determines that there is no identical person. Theidentical person determination unit 3 may estimate an identical personby comparing either the positions or sizes of faces, or may estimatefaces having highest similarities of their luminance values and colorinformation as an identical person. In this manner, by checkingcorrelation between faces detected from two frames, whether or not facesdetected from these two frames are that of an identical person can bedetermined. If image data input to the image input unit 1 is a stillimage, this step S103 is omitted.

In step S104, the face detection unit 2 calculates feature dataincluding coordinates of the features, as shown in FIGS. 6 and 7, fromthe faces that can be detected.

The facial expression determination unit 4 determines in step S105 usingthe feature data calculated by the face detection unit 2 and thedetermination result obtained by the identical person determination unit3 if each face is a “blankness” face. In this embodiment, when thechange amount of the shape of the mouse does not exceed a predeterminedthreshold during a predetermined period of time, the facial expressiondetermination unit 4 determines that a face at that time is a“blankness” face, and determines a “smile” face and the like based onthe feature data of this “blankness” face. More specifically, the facialexpression determination unit 4 accumulates the feature data of facesdetermined as an identical person, and observes a relative change of thefeature data, thereby determining whether or not that face is a“blankness” face. When the facial expression determination unit 4 candetermine a “blankness” face, it calculates feature data used as“blankness” criteria from the “blankness” face. If the feature data arenot sufficiently accumulated, and the facial expression determinationunit 4 cannot calculate feature data used as “blankness” criteria (NO instep S106), the process returns to step S101 via steps S111 and S112 soas to accumulate face feature data. If the facial expressiondetermination unit 4 can calculate feature data used as “blankness”criteria in step S105 (YES in step S106), the process advances to stepS107. Note that the facial expression determination unit 4 may skip theprocess in step S105 for the face for which the feature data used as“blankness” criteria can be calculated until it loses sight of thatface. Alternatively, even after the feature data used as “blankness”criteria are calculated, every time new feature data are calculated instep S104, the facial expression determination unit 4 may calculate newfeature data used as “blankness” criteria, and may update the featuredata.

In step S107, the facial expression determination unit 4 calculates afacial expression of each face indicated by latest feature datacalculated in step S104 by comparing the latest feature data with thoseused as “blankness” criteria calculated in step S105.

In step S108, the database selection unit 5 selects and reads out, fromthe personal database unit 8, all feature data for personalauthentication corresponding to the facial expression determinationresult output from the facial expression determination unit 4. Forexample, if the facial expression determination unit 4 determines a“smile” face in step S107, the database selection unit 5 selects andreads out feature data A-2, B-2, and C-2 of all persons in case of a“smile” face from those in the personal database unit 8 shown in FIG. 2.When feature data of a large number of persons are registered inadvance, the database selection unit 5 may select only feature data of aspecific person designated by the user or those included in a persongroup.

In step S109, the authentication unit 6 identifies who is a person withthat face, based on the absolute value sum S of the latest feature data,the facial expression of which has been determined, and those forpersonal authentication corresponding to that facial expression.

In step S110, the result output unit 7 receives the personalauthentication result from the authentication unit 6, and displays thereceived result by superimposing it on an image generated from imagedata to be authenticated. Of course, the display method of theauthentication result is not limited to such specific display method,and various methods are available. When the personal authentication isexecuted at the same time for a plurality of faces, it is desirable toclarify the correspondence between the authentication results and faces.

If the image data input to the image input unit 1 still includes frameimages which are to undergo face detection (NO in step S111), theprocess returns to step S101 via step S112; otherwise (YES in stepS111), this flowchart ends.

According to the aforementioned embodiment, even when a facialexpression changes, that facial expression is detected, and personalauthentication is executed using feature data corresponding to thedetected facial expression, thus allowing personal authentication withhigh precision.

Second Embodiment

As the second embodiment, a case will be explained wherein feature datacorresponding to facial expressions detected in the first embodiment arenot registered in the personal database unit 8. A personalauthentication apparatus of this embodiment additionally registers, inthe personal database unit 8, feature data of a facial expressiondifferent from that upon authentication using the personalauthentication results of image data of other frames.

FIG. 4 shows the arrangement of the personal authentication apparatus ofthe second embodiment, and a feature data registration unit 9 is addedto the arrangement shown in FIG. 1. For example, assume that featuredata for personal authentication corresponding to “blankness” and “eyeclosure” of a certain person are registered in the personal databaseunit 8, but feature data for personal authentication corresponding to“smile” are not registered. When it is determined that this person has a“smile” face, the feature data registration unit 9 registers featuredata corresponding to the “smile” face of this person in the personaldatabase unit 8 as those for personal authentication corresponding to“smile”.

As for other arrangements, the same reference numerals in FIG. 4 denotethe same parts as in FIG. 1, and a repetitive description thereof willbe avoided.

The operation of the personal authentication apparatus of thisembodiment will be described below with reference to FIGS. 5A and 5B.The processes in steps S113 to S115 in FIG. 5B are different from FIG. 3as the first embodiment. Since the processes denoted by the same stepnumbers as in FIG. 3 of those in FIG. 5A are the same as those in FIG.3, a repetitive description thereof will be avoided.

In steps S101 to S107, the face detection unit 2 detects faces fromrespective frames and calculates feature data of these faces, and theidentical person determination unit 3 determines which of faces of thosedetected from the respective frames correspond to identical persons.Then, the facial expression determination unit 4 determines a facialexpression of a face for which feature data used as “blankness” criteriaare calculated.

The feature data registration unit 9 checks in step S113 whether or notthe face, whose facial expression is determined in step S107, is a facethat has already been personally authenticated. Whether or not a facehas already been personally authenticated can be determined based on theprevious authentication results by the authentication unit 6 and theprevious identical person determination results by the identical persondetermination unit 3. That is, whether or not the identical persondetermination unit 3 can successively track a face, which has alreadybeen personally authenticated based on an arbitrary facial expression inthe previous frame, even after that personal authentication by themethod described in step S103 of the first embodiment is determined. Ifthe face whose facial expression is determined in step S107 is thatwhich has not been personally authenticated yet (YES in step S113), theprocess advances to step S108 to execute the same processing as in thefirst embodiment.

If the face whose facial expression is determined in step S107 is thatwhich has already been personally authenticated (NO in step S113), theprocess advances to step S114. The feature data registration unit 9determines in step S114 whether or not feature data for personalauthentication of that person, which correspond to the facial expressiondetermined in step S107, have already been registered in the personaldatabase unit 8.

If the feature data for personal authentication of that person, whichcorrespond to the facial expression determined in step S107, havealready been registered in the personal database unit 8 (NO in stepS114), the process advances to step S108 to execute the same processingas in the first embodiment.

If the feature data for personal authentication of that person, whichcorrespond to the facial expression determined in step S107, have notbeen registered yet in the personal database unit 8 (YES in step S114),the process advances to step S115.

In step S115, the feature data registration unit 9 registers featuredata calculated from the face with that facial expression as those forpersonal authentication corresponding to that facial expression of theperson in the personal database unit 8.

A practical example will be described below. Assume that feature data of“blankness” of Mr. B have already been registered in the personaldatabase unit 8, but feature data of “smile” have not been registeredyet. Also, assume that it has already been determined based on theauthentication result using the feature data of “blankness” that acertain face is that of Mr. B. Furthermore, assume that the facialexpression determination unit 4 determines a “smile” face for the facedetermined as Mr. B by the identical person determination unit 3 inimage data of a new frame. At this time, the feature data registrationunit 9 newly registers feature data of the face determined as a “smile”face in the personal database unit 8 as those for personalauthentication of a “smile” face of Mr. B. Upon completion ofregistration, this personal authentication apparatus advances to stepS112, and executes the same processing as in the first embodiment.

Note that feature data of respective facial expressions of an identicalperson are extracted and stored, and when that person is authenticated,feature data corresponding to a non-registered facial expression of thestored feature data may be registered as those for personalauthentication.

If the database selection unit 5 determines in step S108 thataccumulation of feature data for personal authentication correspondingto a specific facial expression is insufficient, the result output unit7 may display a message that advises accordingly. When the result outputunit 7 displays this message to the user, it can prompt the user toaccumulate insufficient feature data for personal authenticationcorresponding to that facial expression.

In step S109, the authentication unit 6 may comprehensively authenticatea given face based on not only the personal authentication result forone facial expression but also those for a plurality of facialexpressions. For example, the authentication unit 6 executes personalauthentication using feature data of “smile” for a frame in which afacial expression is determined as “smile” in a movie, using those of“blankness” for a frame in which a facial expression is determined as“blankness”, and those of “eye closure” for a frame in which a facialexpression is determined as “eye closure”. As a result, a person who isdetermined to have the highest degree of coincidence a large number oftimes may be selected. In this way, when authentication results of aplurality of face regions match, a final authentication result isobtained, thus improving the authentication precision. At the time ofexecution of personal authentication for an identical person in a movie,different authentication results may be obtained every time a facialexpression changes. At this time, a “smile” face has larger variationsof the degree of opening of the eyes and that of lift-up of the two endsof the mouse, and has a lower ratio of coincidence with feature data forpersonal authentication compared to a “blankness” face. For this reason,the authentication unit 6 may multiply the personal authenticationresult for a “smile” face by a weight smaller than that in the“blankness” state, and may take statistics of authentication resultsobtained time-serially, thus obtaining a final authentication result.

When different authentication results are obtained even for an identicalperson like that Mr. A is determined in case of “blankness”, and Mr. Bis determined in case of “eye closure”, the feature data for personalauthentication of “eye closure” may be updated using these results. Insuch case, the feature data registration unit 9 updates the feature datafor personal authentication of “eye closure” of Mr. A in the personaldatabase unit 8 using those of “eye closure” which are unwantedlydetermined as Mr. B.

According to the aforementioned embodiment, since feature data which arenot registered in advance are automatically added, personalauthentication with higher precision can be attained.

Other Embodiments

The present invention also includes a case in which the invention isachieved when a computer program that implements the functions of theaforementioned embodiments is directly or remotely supplied to a systemor apparatus. In this case, a computer of the system or the like readsout and executes the computer program.

Therefore, the computer program itself installed in a computer toimplement the functional processing of the present invention using thecomputer implements the present invention. In this case, the form ofprogram is not particularly limited, and an object code, a program to beexecuted by an interpreter, script data to be supplied to an OS, and thelike may be used as long as they have the program function.

As a recording medium (storage medium) for supplying the program, forexample, a flexible disk, hard disk, optical disk, and magneto-opticaldisk may be used. In addition, an MO, CD-ROM, CD-R, CD-RW, magnetictape, nonvolatile memory card, ROM, and DVD (DVD-ROM, DVD-R) may beused.

As another program supply method, the user can establish connection to ahomepage on the Internet using a browser on a client computer, and candownload the computer program itself of the present invention from thehomepage. Also, the computer program can be supplied by downloading acompressed file containing an automatic installation function onto arecording medium such as a hard disk. Also, the computer program thatforms the program of the present invention may be segmented into aplurality of files, which may be downloaded from different homepages.That is, the present invention includes a WWW server which makes aplurality of users download program files required to implement thefunctional processing of the present invention by computers.

Also, a storage medium such as a CD-ROM, which stores the encryptedprogram of the present invention, may be delivered to the user, and theuser who has cleared a predetermined condition may be allowed todownload key information that decrypts the program from a homepage viathe Internet. In this case, the user executes the encrypted programusing the downloaded key information and installs that program on acomputer, thus implementing the present invention.

The functions of the aforementioned embodiments are implemented when thecomputer executes the readout program. In addition, an OS or the likerunning on the computer executes some or all of actual processes basedon an instruction of that program, thus implementing the functions ofthe aforementioned embodiments.

Furthermore, the functions of the aforementioned embodiments areimplemented when the program read out from the recording medium iswritten in a memory of a function expansion board or unit, which isinserted in or connected to the computer, and a CPU or the like on theboard or the like executes some or all of actual processes.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2008-156996, filed Jun. 16, 2008, and No. 2009-105611, filed Apr. 23,2009, which are hereby incorporated by reference herein in theirentirety.

What is claimed is:
 1. A personal authentication apparatus comprising:an input unit configured to input image data; a face detection unitconfigured to detect a face region of a person included in the imagedata input by said input unit, and to detect feature data from thedetected face region; a facial expression determination unit configuredto determine a facial expression from the face region detected by saidface detection unit; a storage unit configured to store feature dataused to authenticate a person in correspondence with respective facialexpressions of a plurality of faces; a selection unit configured toselect feature data corresponding to the facial expression determined bysaid facial expression determination unit from said storage unit; and anauthentication unit configured to authenticate a person by comparing thefeature data of the face region detected by said face detection unit,and the feature data selected by said selection unit.
 2. The apparatusaccording to claim 1, wherein said facial expression determination unitcalculates a change amount of feature data from a plurality ofsuccessive image data, and determines a facial expression of a facebased on the change amount.
 3. The apparatus according to claim 1,wherein said authentication unit authenticates a person for each of aplurality of image data, which are used by said facial expressiondetermination unit in determination of facial expressions of faces, anddetermines a final authentication result based on respectiveauthentication results.
 4. The apparatus according to claim 1, whereinwhen feature data which is not stored in said storage unit andcorresponds to another facial expression of a person is detected from aface, based on which the person is authenticated by said authenticationunit, the feature data is stored in said storage unit as feature datacorresponding to the other facial expression of the person.
 5. Theapparatus according to claim 1, further comprising a unit configured tonotify that feature data corresponding to a facial expression selectedby said selection unit is not stored in said storage unit when thefeature data is not stored in said storage unit.
 6. A personalauthentication method to be executed by a personal authenticationapparatus which includes: an input unit configured to input image data;a face detection unit configured to detect a face region of a personincluded in the image data input by the input unit, and to detectfeature data from the detected face region; a facial expressiondetermination unit configured to determine a facial expression from theface region detected by the face detection unit; and a storage unitconfigured to store feature data used to authenticate a person incorrespondence with respective facial expressions of a plurality offaces, the method comprising: a selection step of selecting feature datacorresponding to the facial expression determined by the facialexpression determination unit from the storage unit; and anauthentication step of authenticating a person by comparing the featuredata of the face region detected by the face detection unit, and thefeature data selected in the selection step.